1. Field of the Invention
The present invention relates to a technique to process an endoscopic image during surgery or examination with an endoscope being inserted in an anatomical structure of a subject, and in particular to a technique to process an endoscopic image with using a virtual endoscopic image that represents an anatomical structure of a subject.
2. Description of the Related Art
In recent years, techniques to observe or treat an anatomical structure, such as the intestine or the bronchial tube, of a patient with using an endoscope are attracting attention.
However, while an endoscopic image provides an image that clearly represents colors and textures of the interior of an anatomical structure captured with an image pickup device, such as a CCD, the interior of the anatomical structure is shown in a two-dimensional image, and therefore it is difficult to understand which position in the anatomical structure the image shows, and it is impossible to obtain information of the depth direction to reach a desired part. To address this problem, techniques to support understanding of the position of the endoscope in an anatomical structure of a subject with using a virtual endoscopic image obtained with a virtual endoscopic technique have been proposed. The virtual endoscopic technique generates an image similar to an endoscopic image from volume data, which is acquired through tomographic imaging with a modality, such as a CT apparatus.
For example, Japanese Unexamined Patent Publication No. 2001-204738 (hereinafter, Patent Document 1) discloses a navigation system for surgery, which involves: detecting the positions of an endoscope and a treatment tool used during surgery with a position detection sensor; measuring the distance from a distal end of an inserted portion of the endoscope to an object at a surgery site; transmitting positional data of the endoscope, etc., relative to reference three-dimensional coordinates and a three-dimensional image or two-dimensional image, which is generated from living body image data prepared in advance, to an image combining circuit; and displaying a three-dimensional image combined with shape data of the endoscope, etc., based on the positional data of the endoscope, etc., and shape data of the endoscope, etc.
Japanese Unexamined Patent Publication No. 2009-254837 (hereinafter, Patent Document 2) discloses a technique for supporting surgery, which involves: providing a position sensor at a distal end of an endoscope; specifying a target position in a reference coordinate system on a virtual endoscopic image; moving and positioning the distal end of the endoscope into a position in an anatomical structure corresponding to a point set in the target position; measuring the position and orientation of a catheter distal end of the endoscope relative to a three-dimensional reference system with a position sensor to align the position of the endoscope with the reference coordinate system to display the catheter distal end and the target on the virtual endoscopic image, for example.
Further, Japanese Unexamined Patent Publication No. 2009-279251 (hereinafter, Patent Document 3) discloses a device which generates a virtual endoscopic image from a three-dimensional image of a subject, that has been acquired in advance, calculates an error between the virtual endoscopic image and a real image captured with an endoscope, shifts the position of a view point of a virtual endoscope and repeats the calculation of the error between the shifted virtual endoscopic image and the endoscopic image until the calculated error becomes equal to or smaller than an acceptable error, and when the calculated error has become equal to or smaller than the acceptable error, calculates the position and orientation of a distal end of the endoscope based on a line of sight parameter of the virtual endoscopic image.
Yet further, Japanese Unexamined Patent Publication No. 2005-334462 (hereinafter, Patent Document 4) discloses a stereoscopic endoscope system for minimizing a misaligned area between right and left images and dissociation of accommodation and convergence, which includes: a stereoscopic hard endoscope for transmitting two subject images of a subject having a parallax therebetween; a focus controller for finding a distance between the subject and the stereoscopic endoscope; a stereoscopic TV camera including a right eye image CCD and a left eye image CCD for imaging the two subject images transmitted from the stereoscopic endoscope and a focal point changing unit for changing the focal point of the subject images; and an imaged area controller for controlling a display area of the subject images in the horizontal direction.
However, while the methods disclosed in Patent Documents 1 to 3 provide understanding of the position of the endoscope in an anatomical structure, it is impossible to apply various known analysis methods and image display methods, such as volume rendering and a display method for displaying a stretched development view of a tubular structure, unlike volume data acquired through tomographic imaging with a CT apparatus, or the like.
Further, while the method disclosed in Patent Document 4 provides three-dimensional understanding of the interior of an anatomical structure, it does not convert the endoscopic images into volume data. Therefore, unlike volume data, it is impossible to use various image analysis methods and image display methods with the endoscopic images.